Stereoscopic electronic microscope workstation

ABSTRACT

A stereoscopic microscope workstation providing high-resolution, real-time video signals to a display means. A stereoscopic microscope workstation providing high-resolution, real-time data to a display means. Various embodiments are disclosed including desktop and free-standing workstations. An image processing unit can be implemented, providing for natural orientation of the magnified image of the viewed object, also allowing rotation, cropping, filtering and other image manipulation features. Methods of performing a procedure utilizing the stereoscopic microscope workstation are disclosed, including a method of performing a procedure of simultaneously utilizing both foveal vision and peripheral vision.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application and claims priority tothe earlier non-provisional application entitled “StereoscopicElectronic Microscope Workstation” filed on this same day, Jan. 29,2007, and having Ser. No. 11/668,400, the disclosures of which arehereby incorporated herein by reference in their entirety. Thisapplication is also a continuation-in-part application and claimspriority to the earlier non-provisional application entitled“Stereoscopic Image Acquisition Device” filed Oct. 21, 2005 and havingSer. No. 11/256,497, the disclosures of which are hereby incorporatedherein by reference in their entirety. This application also claimspriority to the earlier provisional application entitled “StereoscopicElectronic Microscope Workstation” filed Jan. 27, 2006 and having Ser.No. 60/762,577, the disclosures of which are hereby incorporated hereinby reference in their entirety.

TECHNICAL FIELD

This invention generally relates to methods and apparatus forstereoscopic imaging and, more specifically, to a stereoscopicmicroscope workstation.

BACKGROUND

The use of microscopes in the performance of tasks relating tomagnifying small objects is common in many areas of science, medicineand manufacturing. Stereomicroscopes, in which the user sees left-eyeand right-eye views and forms a three-dimensional or stereoscopic view,are also very common. That is, a stereoscopic view provides the normalstereoscopic sense of depth (“stereopsis”) enjoyed by the naturaleyesight of a human user having two eyes, (e.g. normal eyesight).Stereomicroscopes are typically used in scientific research, education,surgery, medical and dental examinations, and industrial inspection andmanufacturing where depth perception or a depth of field of the view isdesirable.

Typically a user sits in an upright position with hands in front atabout waist height for object manipulation. Eyepieces are generallylocated such that the user peers down at an angle into them to see themagnified image of the object. Such a position is generally accepted asergonomic, intended to maximize productivity by reducing user fatigueand discomfort. In fact in some jobs the worker is in this position forhours each workday. However the use of eyepieces in optical systems isoften problematic, particularly in microscopy. Eyepieces require theuser to maintain a fixed position while observing the object or desiredfield of view, such that frequent or prolonged use can lead to eye,neck, and/or back strain. In addition, visualization can be difficultdue to misalignment between eyes and eyepieces, or between eyeglassesand eyepieces, and a significant amount of time is needed to adjust,focus, and clean the eyepieces.

Furthermore, only one user or observer at a time can view imagesgenerated by the microscope and the user can no longer see what ishappening in the surrounding environment. In addition, microscopes, assuch, cannot store images or sequences of images for later reference,process them in special ways, or transmit them to remote viewing sites.Therefore, it is often desirable to use electronic imaging to replacethe eyepiece optics of a microscope.

As noted above, eyepieces require the user to maintain a fixed positionwhile observing the object or desired field of view, such that frequentor prolonged use can lead to eye, neck, and/or back strain. In addition,visualization can be difficult due to misalignment between eyes andeyepieces, or between eyeglasses and eyepieces, and a significant amountof time is needed to adjust, focus, and clean the eyepieces. Frequently,the user using the eyepieces can no longer see what is happening in asurrounding environment.

As is well known in the art, the use of two electronic cameras mountedon a stereomicroscope, each with a slightly different point-of-viewprovided by the microscope's optics can replicate the naturalstereoscopic view perceived by human eyes through the microscope. Inparticular, when the images from the two cameras are displayed on asuitable display device, a stereoscopic, or three-dimensional, or “3D”,image is generated.

In the current art, two independent cameras are typically attached tothe stereomicroscope. The optical path to each camera is made by abeam-splitting element that sends some portion of light from each of thetwo optical paths of the microscope, in the portion of the path betweenthe objective lens or lenses and eyepiece lenses, through theappropriate camera's lens system, to the camera's focal plane while therest of the light continues on to the eyepieces. These cameras can bestill-image capture cameras or moving-image capture cameras.

In the case of video cameras, signals from the two cameras aretransmitted through two or more cables to camera control units (CCU),computers, recorders, or display devices. The image sensors within thecameras are usually of a technology known in the art as charge-coupleddevice (CCD). A filter to reduce the amount of infrared light reachingthe sensor is usually integrated into the sensor assembly and is notremovable.

In the current art, the moving-image cameras are typically standarddefinition (SD) video cameras, that is, cameras that conform to the NTSCor PAL video standards. Unfortunately, the resolution of such standarddefinition video cameras has generally not been adequate to replace theeyepieces entirely. The NTSC and PAL systems suffer from low resolution,poor color fidelity, and motion artifacts (due to the interlaced natureof the raster scan). Imagery from these cameras is not suitable forthose applications, such as surgery, precision assembly, and scientificresearch, which require the highest quality visualization.

Because such systems still generally have the eyepieces, or provisionsfor them, the electronic display cannot be located at the optimallyergonomic position, (e.g. where the eyepieces are located). So thedisplay is generally located off to one side or above the eyepieceline-of-sight. This has the effect that using the electronic displayalone solves some of the eyepiece problems but creates new problems.

The two camera systems described above have further disadvantages.Obtaining and maintaining stereoscopic alignment (necessary forcomfortable, long-term viewing) can be difficult when two independentcameras are mounted on a microscope, each with their own adapters. Thecameras generally protrude from the general body of the microscope andare often mounted in a way that is fragile and prone to breakage.Protruding cameras can interfere with existing microscope knobs andcontrols and other apparatuses in the workspace, limiting possibleinstallation configurations, and their size or position can block theuser's view. Dual camera systems generally require numerous mountingparts, resulting in less reliability and more cost than a single,integrated camera module.

In addition, there are also problems with mounting and connecting thecameras to displays or storage media. The use of two cameras requiresmultiple cables and connectors, resulting in less reliability and moredifficult installation than a single cable/connector arrangement of thepresent invention. The two camera system also typically requires twoCCUs and two storage devices and requires that they be synchronized forbest image quality. Such synchronization significantly increases thecomplexity of the design, capital cost and maintenance of the system.

Such cameras do not allow precise positioning of the imaging sensors toeach other for best stereopsis and comfortable viewing, particularlywhen two off-the-shelf cameras are used. The cameras must beindividually focused after mounting, and, should adjustments such asbrightness and contrast be needed, each camera must be controlledindividually. Where the cameras contain irises, they must also beindividually adjusted for each camera, resulting in the potential forunequal amounts of light entering each camera, which can lead todifficult 3D viewing and eyestrain. All these factors demonstrate thatthe installation and maintenance of such a system can be time-consumingand require a skilled technician.

Image processing is also problematic in such present art systems. Asnoted above, the cameras must be electronically linked in some way sothat the two image streams are synchronized, creating additional cost,vulnerability and complexity. The images that result from the twocameras are generally taken directly to the stereoscopic display device.Should the user require image processing, storage, transmission, ordisplay on alternative displays, additional processing units arerequired for each data stream, creating yet more additional cost,vulnerability and complexity.

Information relevant to attempts to address these problems can be foundin U.S. Pat. No. 5,867,210 and U.S. Patent Application No. 2005/0111088,and at http://www.stereoimaging.com/products/dentimag.html, andhttp://www.leica-microsystems.com/eebsite/products.nsf/allids/ECFFFC6CF17470FEC1257C6D002FBF06(See Digital Photo/Leica IC 3D). However, each one of these attemptedsolutions suffer from one or more of the following disadvantages: (i)the device creates two independent output signals, (ii) the device isnot lightweight or compact, (iii) the device does not provide sufficientimage processing, recording, or transmission capability, (iv) the devicedoes not have adequate resolution in real-time for many applications,(v) the device was designed to be used with eyepieces, or (vi) thedevice is limited with respect to the make or type of opticalinstruments with it which can be used.

SUMMARY

The present invention relates to a compact stereoscopic imageacquisition apparatus and methods capable of acquiring and displayinghigh-quality stereoscopic images. The invention further can be embodiedin compact and reliable workstations providing ergonomic posture for theuse of such stereoscopic devices. More particularly, the disclosedapparatus and method acquire and transfer high-resolution, real-timeimages to image processing, recording, or display systems. The disclosedapparatus and method can perform these desired functions withoutprotruding elements, numerous cables and connectors, and otheradditional components such as eyepieces found in the present art. Asdesired, in other alternate embodiments, the apparatus and methods canbe readily adapted for use with a variety of existing opticalinstruments as well.

Contrary to the tunnel-like circular view (e.g. similar to lookingthrough a tube) typically experienced when using a contemporarystereomicroscope outfitted with eyepieces, the present inventionenhances a targeted visual field and presents this visual field in anatural, three-dimensional view with an ergonomic posture. Further, themagnified image is presented without destroying the user's peripheralview. This desirable feature allows the user to comfortably focus on themagnified image without losing peripheral vision. Preserving the user'speripheral vision allows the user to scan the surrounding workingenvironment without changing focus or foveal view, resulting in a moreeffective and comfortable use of the device as well as a safer and moreconvenient surrounding working environment.

More particularly, a variety of specific aspects of the invention aredisclosed. In one aspect, a stereoscopic microscope workstation for auser to view an object as a magnified image is described. Thestereoscopic workstation primarily comprises of: (i) a stereomicroscopecapable of acquiring a plurality of optical views of the object andproviding a plurality of optical paths of the plurality of optical viewsthereof, (ii) a stereoscopic image acquisition device acquiring theplurality of optical paths from the stereoscopic microscope andtransmitting a real-time image data stream representing the plurality ofoptical paths, and (iii) a display means receiving the real-time imagedata stream and displaying a stereoscopic magnified image of the objectto the user. The stereoscopic microscope workstation is capable ofpresenting the stereoscopic magnified image of the object inhigh-resolution, (at least 1280×720 pixels) for at least two of theoptical views of the plurality of optical views. Where desired, thestereoscopic microscope workstation can display the magnified image ofthe object in the same orientation as the object.

In another aspect, an image processing unit for manipulating thereal-time image data stream prior to the display means receiving thereal-time image data stream can be implemented. Such an image processingunit for manipulating the real-time image data stream provides the userthe ability to rotate, crop, invert, mirror and filter the real-timeimage data stream, among other useful features.

In yet another aspect, the stereoscopic microscope workstation comprisesa single flat LCD display. Where a single flat LCD display is utilizedto present a stereoscopic view of the magnified image of the object,differently-polarized spectacles are configured over the user's eyes toenable a stereoscopic view of the magnified image of the object. Suchspectacles can also have only a portion of the optical lens polarized,such that the remainder of the user's visual field is not opticallymodified by the spectacles, thereby allowing for the performance oftasks with natural vision outside or away from the magnified image ofthe object. Other embodiments are described wherein the display meanscomprises separate left and right views.

In still yet another aspect, the stereoscopic microscope workstation isconfigured in a free-standing configuration.

In another aspect, embodiments of the stereoscopic microscopeworkstation further comprise a deflective element between the object andan objective lens of the stereomicroscope, or further comprise adeflective element to change a central optical axis of the plurality ofoptical paths. In preferred embodiments, such a deflective element thatcan be rotated about one or more axes to provide a change in thelocation of the viewing point of the object in the magnified image.

In certain applications it is preferable to configure a holding meanscapable of holding the object, thereby controlling the position andmotion of the optical views of the object. An example of such a holdingmeans is a stage; such a stage can be configured to quantify therelative distance of points-of-interest on the object to providemeasurement capabilities. Where measurement capabilities are desired buta holding means is not desired or possible, a mounting system capable ofcontrolling the position and motion of the optical views of the objectcan be configured.

In other aspects, methods relating to the invention will also bedisclosed, namely methods providing for: (i) focusing on the magnifiedimage with the user's foveal vision while viewing a manipulation of atool with the user's peripheral vision, (ii) focusing on themanipulation of a tool with the user's foveal vision, while viewing themagnified image of the object with the user's peripheral vision, (iii)focusing on the magnified image with the user's foveal vision andmanipulating the object, wherein the manipulation of the object isviewed in high-resolution in real-time in the magnified image.

While preferred embodiments of the apparatus and methods describedcontemplate medical surgery as an exemplary application, the presentinvention as disclosed can be utilized in a variety of industries,products and professions.

It is to be understood that the details of the various embodiments andaspects of the present invention can be implemented in any combinationwithout departing from the spirit and scope of the invention.

The foregoing and other features and advantages of the present inventionwill be apparent from the following more detailed description of theparticular embodiments of the invention, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a top and side perspective view of the stereoscopic imageacquisition device according to an embodiment of the present invention.

FIG. 1B is a bottom and side perspective view of the stereoscopic imageacquisition device of FIG. 1A.

FIG. 2 is a schematic diagram of the optical principles of a typicalstereomicroscope having eyepieces.

FIG. 3 is a schematic diagram of the optical principles of a typicalstereomicroscope in which the eyepieces are replaced by the stereoscopicimage acquisition device according to the present invention.

FIG. 4A is a top and side perspective view of a device having a straightpattern for the optical paths of the device according to an embodimentof the present invention.

FIG. 4B is a top and side perspective view of a device having a patternhaving one fold for the optical paths of the device according to anembodiment of the present invention.

FIG. 4C is a top and side perspective view of a device having a patternfor the optical paths of the device having two folds according to anembodiment of the present invention.

FIG. 5 is a schematic diagram of the optical principles of a typicalstereomicroscope in which the eyepieces typically attached to themicroscope have been replaced by a stereoscopic image acquisition deviceof the present invention, and where eyepieces have been attacheddirectly to the device.

FIG. 6 is a schematic diagram of an embodiment of the present inventionwhere optical properties of a microscope are built into the device.

FIG. 7 is a schematic diagram of single data stream architecture forprocessing, storing, and displaying digital stereoscopic image data inreal-time according to an embodiment of the present invention.

FIG. 8 is a perspective view of an embodiment in which a deflectingelement is installed between an object and an objective lens of aninstrument to which a device of the present invention is attached andwhere a stereoscopic display is attached to the device.

FIG. 9 is a perspective view of an embodiment of the present inventionillustrating a desktop stereoscopic imaging workstation utilizing astereoscopic display means.

FIG. 10 is a perspective view of an embodiment of the present inventionillustrating a desktop stereoscopic imaging workstation utilizing asingle flat LCD display as a stereoscopic display means.

FIG. 11 is a perspective view of an embodiment of the present inventionillustrating an inverted stereomicroscope workstation utilizing astereoscopic display means.

FIG. 12 is a perspective view of an embodiment of the present inventionillustrating a desktop stereoscopic imaging workstation utilizingseparate left and right views with a barrier as a display means.

FIG. 13 is a perspective view of an embodiment of the present inventionillustrating a free-standing stereoscopic imaging workstation utilizinga stereoscopic display means.

FIG. 14 is a perspective view of an embodiment of the present inventionillustrating a free-standing stereoscopic imaging workstation utilizinga single flat LCD display as a stereoscopic display means.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In the following description, for the purposes of explanation, specificdetails are set forth in order to provide a thorough understanding ofthe invention. However, it will be apparent that the invention can bepracticed without these specific details. In other instances, well-knownstructures and devices are depicted in block diagram form in order toavoid unnecessary obscuring of the invention. Section titles andreferences appearing within the following paragraphs are intended forthe convenience of the reader and should not be interpreted to restrictthe scope of the information presented at any given location.

Various aspects and features of example embodiments of the invention aredescribed in more detail hereinafter in the following sections: (i)Definitions, (ii) Functional Overview, (iii) Stereoscopic ImageAcquisition Device, (iv) Display Means, (v) Stereoscopic ElectronicMicroscope Workstation and (vi) Conclusion.

I. Definitions

Before addressing details of embodiments described below, some terms aredefined or clarified. As used herein, the terms “comprises,”“comprising,” “includes,” “including,” “has,” “having” or any othervariation thereof, are intended to cover a non-exclusive inclusion. Forexample, a process, method, article, or apparatus that comprises a listof elements is not necessarily limited to only those elements but caninclude other elements not expressly listed or inherent to such process,method, article, or apparatus. Further, unless expressly stated to thecontrary, “or” refers to an inclusive or and not to an exclusive or. Forexample, a condition A or B is satisfied by any one of the following: Ais true (or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

Also, use of the “a” or “an” are employed to describe elements andcomponents of the invention. This is done merely for convenience and togive a general sense of the invention. This description should be readto include one or at least one and the singular also includes the pluralunless it is obvious that it is meant otherwise.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Although methods and materialssimilar or equivalent to those described herein can be used in thepractice or testing of the present invention, suitable methods andmaterials are described below. All publications, patent applications,patents, and other references mentioned herein are incorporated byreference in their entirety. In case of conflict, the presentspecification, including definitions, will control. In addition, thematerials, methods, and examples are illustrative only and not intendedto be limiting.

As used herein, the term “compact” when used to describe a stereoscopicacquisition device is intended to mean a device with physical dimensionssmall enough that manual operation of attached modules, including butnot limited to viewing modules such as eyepieces or imaging modules suchas microscopes, is not significantly restricted.

As used herein, the term “data pathway” is intended to mean thatphysical connection between two devices, such as one or more wires,cables, fiber optics, or wireless connections.

As used herein, the term “data stream” is intended to mean a continualmovement of digital data or data structures within a device or betweendevices, perhaps while undergoing processing, conversion, or formatting.

As used herein, the term “data structure” is intended to mean the formin which digital data are addressed, handled, or manipulated, forexample, as bytes, words, blocks, etc. Typically, digital data areformatted into particular data structures as required by the type oftransmission, processing, or display that the data will undergo.

As used herein, the term “deflecting element” is intended to mean anoptical element which can deflect an optical path either in part or inwhole, including without limitation mirrors, prisms and beam-splitters.

As used herein, the term “display controller” is intended to mean acomponent of an image acquisition or processing system which is designedto convert a logical representation of an image stored in memory to asignal that can be used as input for a display medium, most often amonitor utilizing a variety of display standards. A display controllercan also provide functionality to manipulate the logical image inmemory.

As used herein, the phrase “folding the optical paths” is intended tomean deflecting the path towards an intended direction using one or moredeflecting elements.

As used herein, the term “formatting” when referring to data is intendedto mean structuring or restructuring the digital form of the data priorto transmission, processing, or display.

As used herein, the term “high-resolution” when referring tostereoscopic images is intended to mean at least 1280 by 720 pixels(1280×720) for each left or right view. It is contemplated thatresolutions of three times and eight times this minimum resolution canbe implemented depending on the state of technology for sensors anddisplays and depending on what cost is acceptable. On the other hand,the devices of the present invention can be implemented withoutlimitation with higher or lower resolutions for either one or both ofthe views.

As used herein, the term “high-speed data pathway” is intended to mean adata pathway that is capable of data transmission rates required forreal-time, high-resolution stereoscopic images, typically about 1.5gigabits per second or greater. The transmission rate can be in a rangefrom about one to three gigabits per second for a resolution ofapproximately 1280×720 to 1920×1080 pixels for each of left and rightview. Alternatively, resolution of approximately three times thespecific minimum set forth above can operate with a high-speed datapathway having transmission rates of about ten gigabits per second. Thishigher transmission rate and resolution has the advantage of improvedcolor rendition. An even higher resolution of approximately eight timesthe specific minimum set forth above, which also about matches the humaneye in resolution and color rendition, can operate with high-speed datapathways having transmission rates of approximately several hundredgigabits per second.

As used herein, the phrase “high-speed data stream” is intended to meanthat the data flow is fast enough to enable high-resolution stereoscopicimages to be transmitted in real-time.

As used herein, the term “imaging optics” is intended to mean thoseoptical elements that form an image on the focal plane of a sensor,including without limitation lenses and non-planar mirrors.

As used herein, the term “moving-image” when referring to a type orgroup of images or cameras is intended to mean images captured ordisplayed at a speed sufficient that a human eye perceives thesequential presentation of the images as continuous motion.

The term “optical” is intended to mean of or relating to or involvinglight or optics, including without limitation the use of visibleradiation and non-visible radiation to visualize objects.

As used herein, the term “optical element” is intended to mean a part ofan optical system which deflects, refracts, restricts, focuses,manipulates, mirrors, modifies, filters or has some other intendedeffect on a beam of light including without limitation lenses, prisms,mirrors, and beam-splitters.

As used herein, the phrase “optical imaging module” or “imaging module”is intended to mean a component part of an optical instrument, or acorresponding assembly of component elements, that is required forimaging an object in the optical path of the instrument, includingwithout limitation the body of an optical imaging instrument or amicroscope without the eyepieces.

As used herein, the term “optical instrument” is intended to mean anyoptical system of optical elements capable of generating stereoscopicimages including without limitation microscopes, endoscopes, binoculars,telescopes, and optical imaging modules.

As used herein, the term “optical path” is intended to mean thegenerally central ray in an optical system. Should the system have nocentral ray then the optical path is the general centerline of theaverage of all the rays.

As used herein, the term “optical viewing module” or “viewing module” isintended to mean a component part of an optical instrument, or acorresponding assembly of component elements, that is required forviewing an object in the optical path of the instrument, includingwithout limitation an eyepiece module of a microscope.

As used herein, the phrase “real-time” or “in real time” is intended tomean that the image data is acquired, processed, transmitted, ordisplayed at a sufficiently high data rate and at a sufficiently lowdelay that objects on a display move smoothly without user-noticeablejudder or latency. Typically, this occurs when new images are acquired,processed, and transmitted at a rate of at least about 30 frames persecond (fps) and displayed at about 60 fps and when the combinedprocessing of the system has no more than about 1/30^(th) sec of delay.It is possible that individual components of a system be real-time whileone or more other components are not, in which case the entire systemwould not be a real-time system.

As used herein, the term “sensor” is intended to mean an imaging sensor,that is, a small electronic device (“chip”) which contains an array ofindividual light-sensitive sensors, each of which records a smallelement of the image (or “pixel”).

As used herein, the term “stereoscopic image” or “stereoscopic magnifiedimage” is intended to mean a single image consisting of at least twoviews, one corresponding to a left-eye view, i.e. the left view, and onecorresponding to a right-eye view, the right view.

As used herein, the terms “stereoscopic microscope” and“stereomicroscope” are synonymous and intended to mean a microscopecomprising of at least two views, one corresponding to a left-eye view,i.e. the left view, and one corresponding to a right-eye view, the rightview.

As used herein, a “user” is intended to mean an operator or viewer ofthe apparatus utilized or method performed. A user is not limited torepresenting a single person, and can be any number of persons,operators or viewers utilizing the apparatus or performing the methodsdisclosed herein. Additionally, in other embodiments it is anticipatedthat a “user” can also represent one or more non-human biological bodies(e.g. one or more monkeys) or man-made devices (e.g. one or morerobots), or other devices (e.g. manufacturing quality control devices)utilizing the apparatus or method disclosed herein.

II. Functional Overview

Embodiments of the invention relates to a compact stereoscopic imageacquisition device capable of acquiring stereoscopic images from anoptical instrument, such as a stereomicroscope or imaging componentthereof, and workstations providing a display means in an ergonomicposture for the use of such apparatus.

Generally speaking, devices according to the present invention acquireand transfer high-resolution, real-time image data from stereoscopicstill or moving images, to image processing units, recording devices, orone or more displays. Such devices according to the present inventiontypically perform the desired functions without protruding elements,numerous cables and connectors, and other additional components such aseyepieces, and can be readily adapted for use with a variety of opticalinstruments.

Embodiments of the present invention can also be used with present artoptical instruments. The stereoscopic image acquisition device attachesto the body of an optical instrument, replacing the optical viewingmodule of the instrument. A coupling mechanism is used to attach thedevice to the instrument and to align the device's optical paths withthe left and right optical axes of the optical instrument. The couplingmechanism includes elements on the rigid base of the device whichinterface mechanically with elements on the instrument to automaticallyand rigidly align these axes upon installation of the device on theoptical instrument. The coupling mechanism can be adjustable.

A second coupling mechanism is used to attach an optical viewing moduleto the device. In this embodiment, the user can view the object underthe optical instrument either via an electronic image in a display orvia the viewing module. The second coupling mechanism is adjustable.

The optical viewing module replaced by the device is an eyepiece module.In a further embodiment of the invention, the coupling mechanismautomatically aligns the optical paths of the device with the opticalpaths of the viewing module. The optical instrument can be a microscope,and preferably a stereomicroscope.

The optical properties of the optical instrument are built into thestereoscopic image acquisition device in order to provide the imagesotherwise formed by the optical instrument. Hence, no additional opticalinstrument, or imaging component thereof, is required. In one embodimentof the invention, a coupling mechanism can be used to attach an opticalviewing module to the device. In this embodiment of the invention, theuser can view the scene under the optical instrument either via theelectronic image or via the viewing module.

Embodiments of the present invention can also include one or more of:(1) synchronously controlled, high-resolution, real-time sensors adaptedfor acquiring image data from the left and right stereoscopic imagestransmitted from the optical instrument; (2) an adjustment mechanism foraligning the position of the sensors to the at least one optical path ofthe device; (3) an integrated controller for controlling the at leastone sensor and acquiring, processing and transmitting real-time imagedata; (4) another adjustment mechanism for adjusting the focus of theoptical paths within the device; (5) yet another adjustment mechanismfor adjusting the magnification of the optical paths within the device;(6) adjustable irises which can operate in tandem; (7) a furtheradjustment mechanism for adjusting the irises simultaneously; (8) filtercomponents and (9) a replacement mechanism for replacing or changing thefilter components.

In another embodiment of the invention, the device comprises one sensor.In yet another embodiment of the invention, the device can include two,three, or six sensors. The image data transmitted typically comprises aresolution of at least 1280×720 pixels for each view. Embodiments of theinvention can also comprise a deflecting element can be inserted into anoptical path between an object imaged by the optical instrument and themain objective of the instrument. In some applications, it is alsoadvantageous to include a display controller for converting the imagedata to display signals.

A method according to the present invention provides high-resolution,real-time stereoscopic images from an optical imaging instrument orimaging module to a display or recording device. The method can includeone or more of: (1) combining real-time image data acquired from theleft and right views of the stereoscopic images into a single datastructure, (2) processing the image data, and (3) transmitting asequence of single data structures in real time to a stereoscopicdisplay or storage device. The stereoscopic images can be acquired bythe stereoscopic image acquisition device of the present invention.

Embodiments of the present invention can also include a method fordecreasing the physical dimensions of an optical device that replacesthe optical viewing module of an optical instrument. This method can beachieved by placing an optical element or component before a deflectingelement in the optical path of the device, or an associated opticalinstrument, where an optical image is focused on an image sensor. Thepresent invention can also include a device and implement a method wherethe size and direction of the optical paths is manipulated to furtheroptimize the size and shape of the device. For this method, the opticaldevice that is utilized can be the stereoscopic image acquisition deviceof the present invention.

A stereoscopic image processing system can comprise one or more of: (1)an optical instrument for generating stereoscopic images; (2) astereoscopic image acquisition device for acquiring and transmittinghigh-resolution, real-time stereoscopic image data; (3) an acquisitioncontroller for acquiring, processing, and transmitting the stereoscopicimage data in real time; and (4) a stereoscopic display system fordisplaying stereoscopic images. In some embodiments of the invention,the optical instrument, or an optical imaging module thereof, can becoupled to the device. As desired, such an optical imaging module can beshared with and integrated into the device.

Embodiments of the present invention can also include a displaycontroller integral with the device for converting the image data todisplay signals. The display controller can combine stereoscopic imagedata from the left and right views of a stereoscopic image into a singledata structure and transmit a sequence of such data structures as a datastream to a stereoscopic display means or recording device.

In preferred embodiments according to the present invention, it isdesirable to utilize at least two high-resolution, real-time sensorsadapted for acquiring image data from the left and right views of astereoscopic image which the controller can control simultaneously. Thecontroller component can comprise an optical or wireless high speedpathway, or both.

In certain applications, it is desirable to insert a deflecting elementinto an optical path between an object imaged by the optical instrumentand the main objective of the instrument. The optical instrument can bea microscope or a stereomicroscope.

Having provided a function overview of various applications, specificembodiments and their respective components and characteristics shallnow be discussed in conjunction with the attached figures.

III. Stereoscopic Image Acquitition Device

The present invention relates to a compact, stereoscopic imageacquisition device capable of acquiring stereoscopic images from anoptical instrument such as a stereomicroscope, or a component thereof,and providing high-resolution, real-time data from still or movingstereoscopic images to image processing, recording, and display systems.Embodiments of the present invention typically perform the desiredfunctions without the inconvenience and clutter of protruding elements,numerous cables and connectors, and additional components. Otherfeatures and advantages of the invention will be apparent from thefollowing detailed description, and from the claims.

Attention is now directed to more specific details of embodiments thatillustrate the invention, without limitation. FIG. 1A and FIG. 1Billustrate an embodiment of the invention where the stereoscopic imageacquisition device is coupled to the optical imaging module of anoptical instrument, replacing the viewing module of the instrument.

In the illustrated embodiment, the optical instrument is astereomicroscope 1, the imaging module is the body of the microscope,and the stereoscopic image acquisition device has replaced the originaleyepiece module of the microscope. The compact nature of the device,allowing ready access to the controls of the microscope, is evident. Themounting of the device is not prone to misalignment, due to the rigidityof the baseplate 2, and cables to the device therefore haveinsignificant leverage on the mounts, such that unintended pulling on acable does not result in a significant deflection of the device. In thepictured embodiment, the sensor(s) 3 affix to the baseplate 2 and arealigned to the optical path(s) of the device via an adjustment mechanism4, and this apparatus holds the sensor(s) in alignment with the opticalpath(s) of the device, and consequently, in the alignment with opticalpath(s) of the optical instrument.

The device of the present invention is coupled to the stereomicroscopein such a manner that the device's optical paths are automaticallyaligned with the left and right optical axes of the microscope. Ingeneral, the mechanical coupling on the rigid baseplate 2 duplicatesthat of the eyepiece module that it replaces. In one embodiment, thebaseplate has a generally conical protrusion on the bottom to fit withina mating receptacle found on numerous models of microscope bodies fromvarious manufacturers. In this embodiment, the baseplate 2 is securedwith a single clamping screw 5 to the microscope body and is uniquelypositioned by its alignment feature, a groove that accepts an alignmentpin on the microscope body. Other embodiments of the rigid baseplate 2have an adjustment mechanism for adjusting the distance between theoptical path(s) and an alignment feature so that the alignment can becorrected for varying models of optical instruments. Such mechanisms caninclude without limitation spacers, linkages, slotted holes and otherapparatuses known to those skilled in the art. In the case where thereceptacle in the optical instrument is larger than the optimal size forthe baseplate 2, an embodiment of the baseplate 2 can be adjusted byadding or moving bosses around the outside of the conical protrusion, orby substituting a different size conical protrusion, such that theresultant optical alignment is achieved.

The device requires fewer mounting parts to install on a microscope,resulting in more reliability and less cost than prior art. In oneembodiment, the device can be removed or installed through loosening ortightening of a single thumbscrew, using no tools.

As shown in FIG. 1B, the device can be contained within a housing 6 thatcovers and protects the internal components and can provideelectromagnetic shielding. The device does not protrude significantlyfrom the general body of an optical instrument. Sensors 3 are mounted ina way that is not fragile and prone to breakage. The housing 6 can bedesigned to protect the internal components and to not interfere withexisting microscope knobs and controls. The housing 6 can be designedsuch that cleaning and disinfection can be accomplished easily andliquid ingress prevented so that the system can be used in a hospitaloperating room, clean room, or similar environment.

Within the housing 6 is an adjustment mechanism 7 (shown in FIG. 1A) tofocus the imaging optics 8 (shown in FIGS. 1A and 3) so that each deviceis interchangeable with another without the need to re-focus. Therefore,the device can save valuable time in time-critical situations, such asmedical emergencies, because it does not need to be focused aftermounting the system on a microscope. One preferred mechanism forfocusing is a lens barrel wherein the barrel contains a helical track orthread such that rotation of the barrel results in an axial movement ofthe barrel, changing the distance along the optical path from theimaging optical element to the sensor focal plane, thus achieving focus.Such a system can be motorized such that a user can change the focusremotely. Other focusing mechanisms can include without limitation alens barrel inside a cylindrical bore, such barrel being secured by afastener or a rack-and-pinion device whereby the distance from theimaging optics to the focal plane is adjusted by rotating the pinion,either manually or via a remotely-operated motor.

The device can also contain an adjustment mechanism to change, underuser control, the magnification (or “zoom”) of the optical system. Thesemechanisms, commonly known to practitioners in the optical art, can bemanual or motor-driven.

As stated above, the device also contains an adjustment mechanism 4 toadjust the position of the sensors 3 in order to align them with theoptical axes of the device. This adjustment allows precise individualpositioning of the imaging sensor to match image windows for beststereopsis. Mechanisms for aligning the sensors include withoutlimitation mechanical components including jackscrews, cams, or othercomponents known in the art to precisely move the sensor or a mechanicalcomponent to which the sensor is fastened. Following this alignmentprocedure, the sensor 3 can be fixedly attached to the baseplate 2 orother rigid structure by a clamping screw or other mechanism.

In addition, as shown in FIG. 1B, the device has a replacement mechanism9 to insert or remove filter elements 10 (shown in FIG. 3) relative tothe optical paths. These replacement mechanisms can include withoutlimitation wheel or slider plates holding the filters, which canaccomplish this simultaneously for both optical paths. Filter elements,including without limitation IR blocking, selective color, and othertypes of filters, are thereby inserted or removed together, quickly andsimultaneously, resulting in increased ease-of-use and faster and moreerror-free operation than the prior art. In the case of IR blockingfilters, when they are partially or completely removed, the imagebecomes much brighter because of the high sensitivity of CMOS and CCDsensors to IR radiation. This can be used to great advantage in someapplications, such as in eye surgery, scientific research, and low-lightinspection.

Optionally, as shown in FIG. 1B, the device can also have an irisadjustment mechanism 25 for each optical path, control of which is donein tandem by the user while watching a live image. Irises are adjustedsimultaneously via a mechanical linkage in the device, assuring thatequal amounts of light enter each sensor and providing increasedease-of-use and faster, more error-free operation than in the prior art.Such adjustment mechanisms can be manually operated by the user orremotely operated using a motor or actuator.

As shown in FIG. 1A, the device also contains a controller 12 connectedto the sensor. In one embodiment of the invention, the controller is anacquisition controller which performs the following functions: (1)controlling the electronic functions of the sensors; (2) acquiring andprocessing data generated from a left view and a right view of astereoscopic image detected by the sensors; (3) combining stereoscopicimage data from the left view and the right views into a single datastructure; and (4) transmitting the single data structure in real-timeto at least one other device, such as a display.

FIG. 2 shows the optical elements, including eyepiece optics 13, zoomingoptics 14 and the main objective 15 and optical paths 19 of a typicalstereomicroscope with eyepieces.

An embodiment of the stereoscopic image acquisition device 16 coupled toa stereomicroscope 1 is shown in the schematic diagram of FIG. 3,including imaging optics 8 and deflecting elements 17 for controllingthe optical path and forming images on the one or more sensors 3 of thestereoscopic image acquisition device 16. This embodiment contains twooptical paths, each with generally similar construction and opticalcomponents. Although FIG. 3 shows an embodiment of the stereomicroscope1 having a single main objective 15, alternative embodiments of themicroscope can have more than one main objective for forming the leftand right images of the object 18 being imaged.

The main objective 15 of the stereomicroscope 1 forms images of theobject 18 being imaged, which can be magnified by zooming optics 14along the optical paths 19 of the microscope. Each optical path of thestereomicroscope 1 is aligned to the respective optical path 19 of thedevice 16 by the rigid baseplate 2 which is secured to thestereomicroscope 1 by the clamping screw 5. Each optical path 19proceeds thru an adjustable iris 11, the diameter of which the user canadjust in tandem with the other. Each optical path 19 then proceeds thrua region in which a filter 10 can or cannot be inserted by the user (SeeFIG. 1B). Each optical path 19 then proceeds through imaging optics 8and deflecting elements 17 before forming an image on the sensor 3.Sensor control signals for optimizing sensor acquisition parameters andproviding timing for synchronous acquisition are transmitted on thesensor pathway 20 (See FIG. 7) between the sensors 3 and the acquisitioncontroller 12. Image data corresponding to each view is sent from thesensor 3 back to the acquisition controller 12. Within the acquisitioncontroller 12, the data from each view are processed, combined into asingle data structure, and transmitted over a high-speed data pathway 21(See FIG. 7). The acquisition process repeats continually andautomatically.

The acquisition controller 12 contains one or more electronic circuitsthat control all sensors without the necessity of a separate CCU foreach sensor. The controller 12 can also provide, without limitation, thefollowing functionalities:

-   -   1. Powering the sensors and sending parameter settings to them,        such settings can be stored in memory or input by the user;    -   2. Polling the sensors to determine their settings and verify        proper operation;    -   3. Sending timing signals to the sensors simultaneously to        activate them to generate data;    -   4. Acquiring stereoscopic image data by combining the        simultaneous data from the sensors into a single data structure,        repeating this acquisition at a rate sufficient to display        real-time stereoscopic image to the user;    -   5. Performing formatting and conversion of the raw data stream        into a stream of stereoscopic image data; and    -   6. Transmitting the stereoscopic image data to other devices,        such devices including image processing and storage units,        display units and networks for remote transfer.

Additional optional features and operations of the acquisitioncontroller 12 can include:

b 1. Saving sensor parameters to reset the sensors to the user's desiredsettings or to default values;

2. Adjusting left and right image attributes in a technique known in theart as Automatic Gain Control, applied to both sensors independently ortogether;

-   -   3. Matching left and right image attributes through filters        controlling brightness, contrast and a value known in the art as        gamma, applied to both sensors independently or together;    -   4. Diagnosing problems with the sensors and taking steps to        mitigate problems; and    -   5. Performing white balance calibration to both sensors        independently or together.

In addition, the acquisition controller 12 can also contain circuitry toreduce electromagnetic emissions as well as to reduce susceptibility ofelectromagnetic interference from other electronic devices sources.

In an embodiment where the device replaces eyepieces, the optical path19 no longer needs to couple to eyepieces, so the path can be modifiedby using imaging optics 8, followed by deflecting elements 17 to foldthe path to make the overall product smaller, rather than placing thedeflecting element(s) first, as in the prior art. This has the effect ofreducing the overall optical path length significantly and reducingoverall device size. FIGS. 4A, 4B, and 4C show schematic diagrams ofpossible patterns for the optical paths 19 of the device. In a firstembodiment, the optical paths 19 can be straight, as shown in FIG. 4A.This embodiment can include, without limitation, one imaging sensor 3.In a second embodiment shown in FIG. 4B, the optical paths 19 can haveone fold. The directions of the folded paths do not need to be parallelor orthogonal to the plane that contains the incoming optical paths orto each other, as shown in FIG. 4C. In a third embodiment, the opticalpaths 19 can have two folds, as shown in FIG. 3. In other embodiments,the optical paths can cross each other or can have other straight,folded, or non-planar optical paths, as required for the optimum sizeand shape of the device. Because the optical path from the imagingoptics to the sensors is not restricted by the use of an off-the-shelfcamera, as is typical in the prior art, the sensors, deflecting elementsand imaging optics of the present invention can be easily located withinthe housing. The present invention allows optical paths to be folded,resulting in a smaller product size and one that avoids interferencewith knobs, controls and other external items.

The stereoscopic image acquisition device includes at least one, andtypically two, imaging sensors 3 which can be, without limitation, ofthe type known in the art as complimentary metal-oxide semiconductor(CMOS) sensors. Sensors made with CMOS technology use significantly lesspower than CCD sensors. This simplifies the wiring and power suppliesfor the system and allows a simple housing design to dissipate the heatfrom the sensors and electronics, resulting in less cost and complexitythan the prior art. The sensors do not need to conform to the NTSC orPAL standard; preferably they are high-resolution sensors. That is,their pixel count is much greater than standard video. In one embodimentof the invention, the stereoscopic image data acquired by the sensorshas a resolution of at least about 1280×720 pixels for each left orright view. They can conform to the HDTV standard, and they can be“progressive.” That is, the raster scan can not be interlaced.

The sensors used in the device can be color-image sensors. That is, theyproduce red, green, and blue images for each view. If there are one ortwo sensors in the device, they can be of the “single-chip” variety.That is, color filters can be embedded in the sensor so that colorimages can be derived from them, following the appropriate colorprocessing.

An alternative device for acquiring color images can also be used in thedevice. For example, a device known in the art as a “three-chip” imagingdevice can be used, in which three imaging sensors are used for eachview. A beam splitting prism breaks the optical path into threeidentical copies and passes them off to the three sensors through threeprimary color filters. Each sensor receives the entire image; therefore,each color in the resultant image has the same, full, resolution as thesensors. Three-chip systems yield higher quality pictures, but cost morethan one-chip systems. A stereoscopic image acquisition device based onthe three-chip sensor system would contain six sensors in adual-optical-path embodiment or three sensors in a single-optical-pathembodiment.

In one embodiment, two sensors are used to collect the stereoscopicimage data; the left view is formed on one sensor, and the right, on theother, as shown, for example, in FIG. 3. However, one sensor 3 can alsobe used as shown in FIG. 4A. Left and right views can be acquired on onesensor by either: (i) temporal multiplexing, that is, in time, the leftimage is first presented to the sensor and then the right and so on, or(ii) spatial multiplexing, that is, the individual sensing elements ofthe sensor are shared between the two views. In the latter case, leftand right views can be formed on a single sensor by, for example, (a)putting the left view on one side and the right on the other, (b)putting one view on the top half and the other on the bottom, (c)interleaving the two views vertically, (d) interleaving the twohorizontally, and (e) interleaving both vertically and horizontally. Thecase of the stereoscopic image being formed on one sensor 3 in aside-by-side manner is illustrated in FIG. 4A.

In another embodiment of the invention shown in FIG. 5, the device againreplaces the eyepiece module 22 (See FIG. 2) of a stereomicroscope 1;however, an additional coupling mechanism can be provided for an opticalviewing module, typically the eyepiece module 22, to attach to thedevice and for the user to use the eyepieces as usual. Hence, the usercan view the scene under the microscope either via the electronic imageor via the eyepieces. FIG. 5 shows a schematic diagram of the opticalprinciples of the stereoscopic image acquisition device wherein thedevice has replaced the eyepieces of a conventional stereomicroscope 1and the eyepiece module 22 has been attached to the device. The one ormore of deflecting elements 17 of the first embodiment (FIG. 3), whichserved to deflect the entire optical path of the optical instrument inthat embodiment, have been replaced by elements 27 that deflect only aportion of the optical path while allowing the rest of the path to reachthe eyepiece module 22.

In such an embodiment, the eyepiece module 22 typically has a generallysimilar mechanical coupling as that described previously for thebaseplate 2. The framework of this embodiment can then have a similarmating receptacle 23 on the device as described for the baseplate, suchthat the eyepiece module 22 can be automatically coupled to and fixedlyaligned with the optical paths of the device.

In a further embodiment of the invention shown in FIG. 6, the opticalelements of a stereoscopic optical instrument are built into the device.FIG. 6 thus shows a schematic diagram of the optical principles of theintegrated device wherein the optical elements of a stereomicroscope,including zooming optics 14 and the main objective 15 as shown in FIG.3, are now incorporated into the device. Thus this embodiment is astandalone device that does not require a stereomicroscope or an imagingmodule in order to acquire stereoscopic images.

In a further embodiment of the invention (not shown), the opticalelements of an optical instrument are again built into the device.However, a coupling mechanism, as described previously in the secondembodiment (FIG. 5), can be provided for the fitting and usage ofeyepieces on an integrated device similar to that shown in FIG. 6.Hence, the user can view the scene stereoscopically either via theelectronic image or via the eyepieces.

As shown in FIG. 7, digital image data can be continuously streamed toexternal devices, such as processors, computers, or display systems in astereoscopic image processing system 28, via a high-speed data pathway21. This high speed data stream is transmitted in real-time by one ofseveral mechanisms. For example, one mechanism utilizes a plethora ofsignal lines transmitting in parallel to achieve the high data rate.Another mechanism utilizes one or more coaxial cables transmitting datain a generally serial manner. Another mechanism utilizes a wirelessconnection, that is, radio or other frequency transmission andreception. Another mechanism utilizes optical fiber wherein a laser canbe modulated at sufficient frequency and in such a manner as to transmitthe data at very high data rates.

IV. Display Means

The stereoscopic display means 24 of FIG.7 can be, without limitation,one of several types. In a dual projector embodiment, each projectorprojects through a polarizer, with the two polarizers orientedorthogonal to each other. A user can wear orthogonally polarizedspectacles. With such polarized spectacles being oriented to block theinappropriate polarized image and transmit the appropriate image, theright eye will see only the right image and the left eye will see onlythe left image and stereopsis is achieved. The differently-polarizedspectacles can also be configured to have other differing opticalproperties, (e.g. different wavelengths/colors, etc.) to achieve thesame desired result.

A second type of display comprises two LCD displays mounted obliquely toeach other and a beam splitter mounted at the plane bisecting the anglebetween the two LCDs. The LCDs have polarizers such that stereopsis isachieved for a user wearing polarized spectacles as in the projectorembodiment described above.

A third type of display, know in the art as a Head Mounted Display orHMD, comprises left and right small imaging displays and suitable opticsmounted in a rigid spectacle-like frame such that each display presentsits image to the appropriate eye of the user and stereopsis is achieved.

A fourth type of display comprises a fast-switching display thatswitches between left and right images. The user wears specialspectacles containing shuttering devices in front of each eye such thatthe right eye shutter closes when the left image is presented on thedisplay and similarly for the other eye such that stereopsis isachieved. The switching process is performed at a speed greater than thehuman perception limit such that a flickering image is not observed bythe user.

A fifth type of display comprises the stereoscopic image views, left andright, vertically interleaved line-by-line in the display panel or tube,with a barrier grid or lenticular optical panel overlaid on the displaysuch that the user's eyes can only see the appropriate display when theuser is located in the proper location and stereopsis is achieved.

In a further embodiment of the device or system, a deflecting element 17can be placed in the optical path 19 between the object 18 being imagedand the main objective 15 of the optical instrument 1 as shown in FIG.8. This has the effect of deflecting the orientation of the optical pathfrom generally vertical to generally horizontal, which has the furthereffect of reducing the overall height of the device and the opticalinstrument, allowing a user to more easily see over them. A stereoscopicdisplay means 24 can be attached to the device in such a position that auser manipulating the object 18 manually will have a convenient view ofit. Prior art did not allow the placement of the display in thisposition due to the height of the system; rather the display was locatedto one side or the other or much higher, resulting in an awkward workingposition for the user.

V. Stereoscopic Electronic Microscope Workstation

In practical applications such as the medical profession, preferredembodiments of the present invention typically comprise of astereoscopic image acquisition device or “SIAD”, a stereomicroscope thatcan be attached or built into the device, a stereoscopic display meansas described above and further described below, and an image-processingunit or “IPU.”

In another embodiment the invention relates in a similar manner to asystem using two separate cameras, and a stereomicroscope that can beattached or built into the device, such system that provides a signal orsignals to a stereoscopic display means which can or can not beattached, with or without the use and inclusion of an image processingunit, camera control unit or units, or similar device or devices. In thefollowing descriptions the SIAD could be interchanged with the twocameras to achieve similar results with this embodiment.

More particularly, in the present art a high-resolution, real-timestereoscopic microscope workstation utilizing such multiple camerasdirectly connected to multiple display means has not been evidenced. Assuch, while a stereoscopic image acquisition device remains a preferredembodiment of the present invention, thereby providing numerouspractical advantages, the present invention can also be embodied in aworkstation utilizing multiple video signals from multiple video cameraswithout the use of a stereoscopic image acquisition device rendering asingle image data stream. Embodiments of the present invention,therefore, are also directed to the ergonomic design of a stereoscopicmicroscope workstation utilizing more traditional video signal captureand transmission technologies.

Preferably, an image processing unit is attached to the workstationresulting in a complete, one-piece apparatus that provides all thenecessary functionality including means for holding an object, forming astereoscopic electronic image of it, processing such image data anddisplaying a possibly magnified, unmagnified, or de-magnifiedstereoscopic image of the object, in a desired orientation and condition(e.g. not inverted or reflected, or in any orientation desired by theuser), on the stereoscopic display in a convenient position for theuser, in real time. In other embodiments some or all of the functions ofthe IPU can be built into circuitry, firmware and software inside theSIAD or elsewhere in the system such that a separate IPU component cannot be required, possibly reducing the size and cost of the system. Inyet other embodiments, where physical size of the image processing unitis a constraint, the image processing unit can be separately located inan alternate desired location, such as an central computer room or othernon-sterile location.

Control of the system can be accomplished through a separate userinterface device such as, without limitation, pushbuttons or otherswitching devices, a touchpad screen, separate or attached or a part ofthe 3D display, or a joystick, mouse or similar device controlling thesystem that provides user feedback on the 3D display screen or onanother screen or output indication device. Such user feedback could bestereoscopic to enhance effectiveness.

Communication, including transmittal of image data, and additionalcontrol of a system of these embodiments could be performed thru anexternal interface, including without limitation a network, USB,“firewire” or custom designed port, to other devices or to a network.

FIGS. 9 through 13 illustrate various embodiments of a stereoscopicelectronic microscope workstation, along with various components andcharacteristics thereof.

Turning to FIG. 9, an object 31 under observation by a user 38 isillustrated within a central optical axis 32. Upon deflection into amain objective lens located inside the stereomicroscope 35 the centralaxis 32 has a generally horizontal portion in this embodiment. With thisembodiment and its single objective lens the central optical axis israther obvious. In embodiments having no objective lens, multipleobjective lenses or multiple optical paths the central optical axis isdefined as the general central axis of all pertinent axes or bodies.

As depicted, a holding means 33 can be utilized to stabilize the object31 under observation, where required. Examples of holding means 33 arean X-Y stage, as illustrated, slides or other physical, chemical ormagnetic means to stabilize object 31 in a position and orientation foreffective viewing. A stereoscopic image acquisition device 34, astereomicroscope 35, a stereoscopic display 36 and an image processingunit 37 is also illustrated in the figure.

In the embodiment illustrated in FIG. 9 a beam-splitting styledeflective element 39 is utilized. A deflecting element mounting system40 as illustrated provides a manual mechanism providing 2-axis motioncontrol for the deflecting element 39.

After processing by the stereomicroscope 35, the stereoscopic imageacquisition device 34 and the image processing unit 37, a magnifiedimage 41 of object 31 is presented in the display means 36.

It is also desirable in preferred embodiments to equip thestereomicroscope 35 with a laser pointer 42 for positioning andorienting the object 31 on the holding means 33 under thestereomicroscope 35.

It is noted that the image received by the stereomicroscope 35 is areflection of the object 31. However, the desired magnified image 41 canbe configured to be presented in the same orientation as the object 31,such that a user's movement of a tool at the object 31 is shown in asimilar direction and orientation at the magnified image 41. Since theacquired image data is electronic, such image data can be reversed orde-reflected by circuitry inside the stereoscopic image acquisitiondevice 34 or the image processing unit 37 (or a display controller, ifconfigured), to provide a desirable, non-reflected magnified image.Alternatively, one or more additional units of a deflecting element 39can be placed in the optical path, or display means 36, to re-reflectthe magnified image 41.

A further embodiment of the invention uses a display means 36 placed ata distance from the user 38 such that when the user 38 focuses on themagnified image 41, the peripheral vision of the user 38, including anyreduction of it due to spectacles required for 3D viewing, allows theuser 38 to see the object 31, a hand tool or other pertinent object ormotion of them without losing focus on the magnified image 41.Alternatively, the user 38 could focus on the object 31 being imaged ortool and still see the magnified image 41 through peripheral vision.Though peripheral acuity is not as good as foveal acuity it can besufficient to allow the user 38, for example, without limitation, tochange a hand tool while not losing focus on the magnified image 41. Forexample, a medical surgeon performing a procedure utilizing anembodiment of the present invention could change a hand tool from ascalpel to a clamp utilizing peripheral vision without losing focus onthe magnified image 41.

Conversely such a system, where the display means 36 is configured in anoptimal position, allows the user 38 to focus, for example, on a tooland generally see the magnified image 41 on the display means 36 withperipheral vision such that motion of the object 31 can be seen. Theseare important improvements to the art, especially in situations wherelong periods are spent doing meticulous work using a variety ofdifferent tools under the stereomicroscope 35, such as in eye surgery.Such a system reduces tool-change time and fatigue of the user 38.

The position of the display means 36 can also be configured to allow formultiple users desiring to observe the magnified image 41. Shouldadditional display means 36 be desired, such as for a remote location,the digital image data can be replicated to additional locations oranalog electronic splitting devices can be installed to duplicateelectronic signals, (if the display means utilizes an analog signal).The display means 36 shown in FIG. 9 can also include side shields (notshown) that block reflections from ceiling or overhead lights into theeyes of the user 38.

To accommodate the height or personal preference of a user 38, it can bedesirable to move the stereoscopic image acquisition device 34 andstereomicroscope 35 closer to or farther from the display means 36, orto tilt them such that the central optical axis 32 is no longergenerally horizontal, or to move or tilt the display means 36, tooptimize viewing distance and avoid display occlusion. A mechanism to dothis could also be attached to the apparatus. Such a mechanism caninclude the ability to move the deflecting element 39 simultaneously tokeep a desired view of the object 31, for example by rotating thestereoscopic image acquisition device 34 and stereomicroscope 35 aboutthe reflection point of the central optical axis 32 while rotating theelement by one half of the stereoscopic image acquisition device 34 andstereomicroscope 35 rotation.

In a further embodiment of this invention, the stereomicroscope 35 andstereoscopic image acquisition device 34 are placed above the line ofsight of the user 38, as shown in FIG. 10. For simplicity the means tohold the object is not shown in this figure. In this embodiment thedeflecting element 39 is not necessarily required to clear theline-of-sight and can be omitted if desired. This embodiment allows theuser 38 to see the object 31 and its magnified image 41 with only asmall change in viewing angle.

As shown, the user 38 is wearing a pair of polarized spectacles 46 toview the single flat LCD display means 36 with stereoscopic vision. Ifsuch spectacles 46 are utilized for stereoscopic viewing, the bottomportion of the lens and frame of a spectacles 46 can optionally beremoved (not as illustrated) to allow an unobstructed, generallydownward, view of the object 31 while still providing the threedimensional view when looking, generally straight, at the display means.

Turning to FIG. 11, a further embodiment of a stereoscopic microscope ina workstation, positions the microscope below the object. Such anorientation of the microscope can also be termed an “invertedmicroscope” in the art.

A further embodiment, as shown in FIG. 12, positions the displaydirectly in front of and very close to the user 38. This display meanscould be of the barrier type, where a separate left view 47 and rightview 48 are presented side-by-side with a barrier 49 to block thecross-eye view and possibly including optics to present the magnifiedimage 41 to the eye for proper viewing. While not illustrated, a displaymeans similar to that known as the “head mounted display” could also beutilized, whether the display means is mounted to the stereomicroscope35 or the head of the user 38. The user 38 positions himself or herselfto see the magnified image 41 optimally. These embodiments have aneffect of reducing the physical dimensions of the workstationconsiderably.

Alternatively, other types of display means can be used in thisworkstation and in conjunction with the stereoscopic image acquisitiondevice 34. For example, these could include a fast-switching visualdisplay device that switches between left and right images and includesa switching device such that the user 38 wears differently-polarizedspectacles, (not illustrated). Another type of display means is one inwhich a single flat LCD is used and the left and right images areinterleaved horizontally and polarized differently such that the userwears differently-polarized spectacles, (not illustrated).

Turning briefly to FIG. 14, a preferred single flat LCD display means 36is illustrated. While there are a plurality of such display meanscommercially sold, one such display means is sold under the trademark“PolarScreen.” In this embodiment, two flat liquid crystal display (LCD)panels are stacked one on top of each other to form a single flat LCDdisplay, wherein the first LCD panel controls total pixel intensity andthe second LCD panel controls left-eye/right-eye distribution ratio. Insuch an embodiment, a user 38 typically wears differently-polarizedspectacles 46. The readily observable benefit of such an embodiment isthat it provides a stereoscopic display means 36 as a single flat LCDdisplay without the clutter of multiple screens shown in otherembodiments illustrated herein.

Speaking generally to embodiments illustrated in FIGS. 9 through 13,along with other embodiments contemplated by the present disclosure,additional deflecting elements can be used, both in series and inparallel with respect to the central optical axis or optical axesparallel to it. The following configurations can apply to one or moredeflecting elements configured in an embodiment of the presentinvention.

A deflective element can be mounted such that it can be rotated aboutone or more axes to provide a change in the location of the viewingpoint of the object being viewed. This aspect provides a “panning”action, useful in viewing a moving object. Such a feature also providesan easy way to change the view on a heavy or difficult-to-move object.

A mounting can be controlled manually with a mechanism by the user orwith a motorized system and joystick or similar I/O device. Additionallya computer can cause the motion to follow a pre-programmed path or avision-recognition system could cause it to track a moving object. Sucha mounting system can also be configured to control multiple deflectingelements to move differently from each other or the same as each other.

Optionally, the position and motion of such a mounting system could bequantified and the relative distance and speed of points-of-interest onthe object could then be extrapolated to provide measurement capability.

A deflective element could be mounted to allow motion, with respect tothe rest of the device, generally horizontally along the central opticalaxis (i.e. to/from the user) to change vertical working distance intohorizontal and vice-versa.

Alternatively, a deflective element could be fixed with respect to theobject, and the stereomicroscope and stereoscopic image acquisitiondevice portion of the apparatus could be moved generally horizontallyalong the central optical axis (i.e. to/from the element) to provide ameans to adjust the focus of the object's image on the sensors' focalplanes. In such an embodiment the deflective element can have anadjustable mounting system to precisely align the optical axis to thefocus axis such that the view does not move transversely when the focusis changed.

In yet other applications, the stereoscopic image acquisition device andstereomicroscope components, (with or without the deflecting element),can be rotated such that a view of the object from a desired directionis obtained, and the central optical axis may or may not be generallyvertical. The displayed image can be oriented as desired by the user byelectronically modifying the image data in the image processing unit, orpossibly by rotating the display means.

A deflective element can also be mounted to allow quick replacement withother deflective elements having different optical or other properties.Quick replacement can also be useful for quick cleaning or maintenanceof the deflective element.

It is also contemplated that deflective elements mounted at differentangles to the central optical axis could be used to provide optimalangles of the axis for different applications. The mounting angle couldbe adjusted in the mount and then fixed at an optimal angle.Additionally, the body of the device including the stereomicroscope andstereoscopic image acquisition device can be configured such that thecentral optical axis, after deflection by the deflecting element, is nothorizontal.

A deflecting element can have optical properties known as a “coldmirror” that reflects visible light but transmits infrared light.Alternatively, it can likewise have properties known as a “hot mirror”that reflects infrared light but transmits visible light.

A deflective element can have optical properties that transmit a rangeof wavelengths and reflect other wavelengths as desired. Such adeflective element can be useful, for example, for fluorescence imagingwhere an object is illuminated with one wavelength, causing it tofluoresce, and viewed in other wavelengths which show fluorescing of theobject. The deflecting element could pass the illumination beam butreflect, into the microscope, the reflected light of the fluorescingwavelength.

A deflective element could be planar or non-planar.

A deflective elements shape could be a portion of circular cylinder,spherical, parabolic, ellipsoidal or of another non-planar shape suchthat the element can focus an image on the sensors of the stereoscopicimage acquisition device without the use of refractive optics. Such adeflective element could be useful, for example, for imaging in infraredwavelengths.

A deflective element can be configured as a beam-splitter, and candeflect a portion of the light to a different axis. An illuminator canbe placed above or near the deflective element causing generallycollimated light to illuminate the object in an axis generally coaxialwith the central optical axis. This configuration allows the illuminatorto be truly coaxial and to be generally easily adjusted for alignment.

A deflective element can deflect a portion of the light. A laser pointeror similar device can be placed above the deflective element causing agenerally downward vertical beam of laser light to be generally coaxialwith the central optical axis, illuminating a spot on the object suchthat the area being imaged can be observed or its reaction to laserlight can be observed. Such a configuration allows quick placement of anobject such that a desired viewing area of an object can be properlypositioned. Alternatively, the pointing device could be made toilluminate along the horizontal portion of the central optical axis andthe deflecting element could then deflect its beam in a similar mannerto designate the area on the object.

A device known in the art as a stage 33 can be utilized to hold theobject being imaged, as shown in FIGS. 9 and 12. Motion and position ofthe stage's platen could be quantified and the relative distance andspeed of points-of-interest on the object could then be extrapolated toprovide measurement capability.

In an alternative embodiment the deflecting element could be placedbetween the main objective lens of the stereomicroscope and the zoomingor imaging optics such that the central optical axis is again deflectedfrom being generally vertical to being generally horizontal, with theaxis of the main objective being generally vertical.

Turning to FIGS. 13 and 14, a stereoscopic microscope workstation in afree-standing configuration is illustrated. As depicted, it can beuseful to mount a stereoscopic microscope workstation on an articulatingarm 43, possibly with a device to counterbalance the weight andlock/unlock the arm's joints 44. The stereomicroscope 35 and displaymeans can be easily positioned in the appropriate location for the taskat hand. For example, without limitation, a wheeled pedestal 45 with thearticulating arm 43 and joints 44 can be easily maneuvered and thecentral optical axis 2 properly positioned such that a user 38 could usethis workstation to perform surgery on a patient located on a holdingmeans 33. The central optical axis 32 need not be generally vertical insuch embodiments. Rather the central optical axis 32 could be orientedin any desired direction to obtain the desired view of the object, (e.g.the side of the patient on a horizontal axis).

In other embodiments, the wheeled pedestal 45 can be part of an existingdevice, as in for example a surgical microscope, and the othercomponents of this invention could be attached by means of retrofit toit. The workstation can also be mounted from the ceiling or from anoverhead gantry rather than the wheeled pedestal 45 illustrated.Alternatively, such an overhead gantry, or an additional gantry, can bemounted between the display means 36 and stereoscopic image acquisitiondevice and stereomicroscope components to move them with respect to eachother.

VI. Conclusion

The embodiments and examples set forth herein were presented in order tobest explain the present invention and its practical application and tothereby enable those of ordinary skill in the art to make and use theinvention. However, those of ordinary skill in the art will recognizethat the foregoing description and examples have been presented for thepurposes of illustration and example only. The description as set forthis not intended to be exhaustive or to limit the invention to theprecise form disclosed. Many modifications and variations are possiblein light of the teachings above without departing from the spirit andscope of the forthcoming claims. For example, the device could beadapted slightly for use in other type of optical viewing systemsincluding, without limitation, binoculars and endoscopes.

Unless otherwise indicated, all numbers expressing quantities ofingredients, properties such as molecular weight, reaction conditions,and so forth used in the specification and claims are to be understoodas being modified in all instances by the term “about” or“approximately.” Accordingly, unless indicated to the contrary, thenumerical parameters set forth in the following specification andattached claims are approximations that can vary depending upon thedesired properties sought to be obtained by the present invention. Atthe very least, and not as an attempt to limit the application of thedoctrine of equivalents to the scope of the claims, each numericalparameter should at least be construed in light of the number ofreported significant digits and by applying ordinary roundingtechniques. Notwithstanding that the numerical ranges and parameterssetting forth the broad scope of the invention are approximations, thenumerical values set forth in the specific examples are reported asprecisely as possible. Any numerical value, however, inherently containcertain errors necessarily resulting from the standard deviation foundin their respective testing measurements.

The terms “a” and “an” and “the” and similar referents used in thecontext of describing the invention (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural, unless otherwise indicated herein or clearly contradicted bycontext. Recitation of ranges of values herein is merely intended toserve as a shorthand method of referring individually to each separatevalue falling within the range. Unless otherwise indicated herein, eachindividual value is incorporated into the specification as if it wereindividually recited herein. All methods described herein can beperformed in any suitable order unless otherwise indicated herein orotherwise clearly contradicted by context. The use of any and allexamples, or exemplary language (e.g. “such as”) provided herein isintended merely to better illuminate the invention and does not pose alimitation on the scope of the invention otherwise claimed. No languagein the specification should be construed as indicating any non-claimedelement essential to the practice of the invention.

Groupings of alternative elements or embodiments of the inventiondisclosed herein are not to be construed as limitations. Each groupmember can be referred to and claimed individually or in any combinationwith other members of the group or other elements found herein. It isanticipated that one or more members of a group can be included in, ordeleted from, a group for reasons of convenience and/or patentability.When any such inclusion or deletion occurs, the specification is hereindeemed to contain the group as modified thus fulfilling the writtendescription of all Markush groups used in the appended claims.

Preferred embodiments of this invention are described herein, includingthe best mode known to the inventors for carrying out the invention. Ofcourse, variations on those preferred embodiments will become apparentto those of ordinary skill in the art upon reading the foregoingdescription. The inventor expects skilled artisans to employ suchvariations as appropriate, and the inventors intend for the invention tobe practiced otherwise than specifically described herein. Accordingly,this invention includes all modifications and equivalents of the subjectmatter recited in the claims appended hereto as permitted by applicablelaw. Moreover, any combination of the above-described elements in allpossible variations thereof is encompassed by the invention unlessotherwise indicated herein or otherwise clearly contradicted by context.

Furthermore, numerous references have been made to patents and printedpublications throughout this specification. Each of the above citedreferences and printed publications, if any, are herein individuallyincorporated by reference in their entirety.

In closing, it is to be understood that the embodiments of the inventiondisclosed herein are illustrative of the principles of the presentinvention. Other modifications that can be employed are within the scopeof the invention. Thus, by way of example, but not of limitation,alternative configurations of the present invention can be utilized inaccordance with the teachings herein. Accordingly, the present inventionis not limited to that precisely as shown and described.

1. A stereoscopic microscope workstation for a user to view an object asa stereoscopic magnified image, comprising: a stereomicroscope capableof acquiring a plurality of optical views of the object and providing aplurality of optical paths of the plurality of optical views thereof;one or more high-resolution cameras for acquiring the plurality ofoptical paths from the stereomicroscope and transmitting one or morereal-time video signals representing the plurality of optical paths,wherein the one or more real-time video signals having a resolution ofat least 1280×720 pixels for at least one of the optical views thereof;and, one or more high-resolution display means receiving the one or morereal-time video signals and displaying a magnified image of the objectto the user, where at least one of the one or more high-resolutiondisplay means having a resolution of at least 1280×720 pixels.
 2. Thestereoscopic microscope workstation of claim 1, further comprising oneor more deflecting elements configured between the object and thestereomicroscope.
 3. The stereoscopic microscope workstation of claim 1,further comprising one or more deflective elements that can be rotatedabout one or more axes to provide a change in the location of theviewing point of the object in the magnified image.
 4. The stereoscopicmicroscope workstation of claim 1, wherein the magnified image of theobject is displayed in the same orientation as the object.
 5. Thestereoscopic microscope workstation of claim 1, wherein the one or morehigh-resolution display means comprises a single flat LCD display, suchthat the user wears differently-polarized spectacles to achieve astereoscopic view of the object in the magnified image.
 6. Thestereoscopic microscope workstation of claim 1, wherein the displaymeans comprises separate left and right views.
 7. The stereoscopicmicroscope workstation of claim 1, wherein the stereoscopic microscopeworkstation is configured in a free-standing configuration.
 8. Thestereoscopic microscope workstation of claim 1, further comprising aspectacles, wherein at least a portion of the spectacles is polarized,thereby providing a stereoscopic view of the magnified image of theobject for the user.
 9. The stereoscopic microscope workstation of claim1, further comprising a holding means capable of holding the object,thereby controlling the position and motion of the optical views of theobject.
 10. The stereoscopic microscope workstation of claim 9, whereinthe holding means is a stage, and wherein the stage can quantify therelative distance of points-of-interest on the object to providemeasurement capabilities.
 11. The stereoscopic microscope workstation ofclaim 1, further comprising a mounting system capable of controlling theposition and motion of the optical views of the object.
 12. Thestereoscopic microscope workstation of claim 11, wherein the mountingsystem can quantify the relative distance of points-of-interest on theobject to provide measurement capabilities.
 13. A method of performing aprocedure upon an object by a user, comprising the steps of: providing astereoscopic microscope workstation, wherein the stereoscopic microscopeworkstation comprises: a stereomicroscope capable of acquiring aplurality of optical views of the object and providing a plurality ofoptical paths of the plurality of optical views thereof; a stereoscopicimage acquisition device acquiring the plurality of optical paths fromthe stereomicroscope and transmitting a high-resolution real-time imagedata stream representing the plurality of optical paths; a display meansreceiving the real-time image data stream and displaying a stereoscopicmagnified image of the object to the user; focusing on the magnifiedimage with the user's foveal vision; and, viewing a manipulation of atool with the user's peripheral vision, while not losing foveal visionfocus on the magnified image of the object.
 14. The method of performinga procedure by a user according of claim 13, wherein the manipulation ofthe tool comprises changing a hand tool.
 15. The method of performing aprocedure by a user according of claim 13, wherein the manipulation ofthe tool comprises positioning the tool closer to the object.
 16. Themethod of performing a procedure by a user according of claim 13,wherein the procedure performed by the user is medical surgery.
 17. Amethod of performing a procedure upon an object by a user, comprisingthe steps of: providing a stereoscopic microscope workstation, whereinthe stereoscopic microscope workstation comprises: a stereomicroscopecapable of acquiring a plurality of optical views of the object andproviding a plurality of optical paths of the plurality of optical viewsthereof; a stereoscopic image acquisition device acquiring the pluralityof optical paths from the stereomicroscope and transmitting ahigh-resolution real-time image data stream representing the pluralityof optical paths; a display means receiving the real-time image datastream and displaying a stereoscopic magnified image of the object tothe user; focusing on the manipulation of a tool with the user's fovealvision, while viewing the magnified image of the object with the user'speripheral vision.
 18. The method of performing a procedure by a useraccording of claim 17, wherein the manipulation of the tool compriseschanging a hand tool.
 19. The method of performing a procedure by a useraccording of claim 17, wherein the manipulation of the tool comprisespositioning the tool closer to the object.
 20. The method of performinga procedure by a user according of claim 17, wherein the procedureperformed by the user is medical surgery.